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Stability of Discrete Time Recurrent Neural Networks and Nonlinear optimization 

problems 

Dr. Nikita Barabanov, and Jayant Singh 

Abstract We consider the method of Reduction of Dissipativity Domain to prove global Lyapunov 
stability of Discrete Time Recurrent Neural Networks. The standard and advanced criteria for Absolute 
Stability of these essentially nonlinear systems produce rather weak results. The method mentioned 
above is proved to be more powerful. It involves a multi-step procedure with maximization of special 
nonconvex functions over polytopes on every step. We derive conditions which guarantee an existence of 
at most one point of local maximum for such functions over every hyperplane. This nontrivial result is 
valid for wide range of neuron transfer functions. 


1 Introduction and Problem Setting 

In this paper, we study existence of points of local maxima for function /( x) = )Tb =1 c i<P( x i)i where 
</)(■) is a nonlinear function, over a hyperplane. This problem arises in stability analysis of nonlinear 
dynamical systems [2], for example Recurrent Neural Networks (RNN). A typical RNN can be described 
by the following systems of equations; 


x k+1 =0(W 1 x k + V n x k +b 1 ), 
x k+1 = (p(W 2 x k + V 1 x\ +1 + b 2 ), 

x k n +1 = 4>(W n x k n + V„-i X k t{ + b n ), (1) 

where n is the number of layers, </>(•) is the activation function, x k is the state vector of the layer j at 
time step k , W :J and Vj are fixed weight matrices, and bj is a fixed vector representing bias. 

Stability of RNN has been addressed extensively in literature. In [4], a stability criterion has been 
developed using NL q approach. A typical NL q system( without external inputs), is of the form, 

Pk +1 = P 1 Q 1 P 2 Q 2 ■ ■ ■ PqQqPk (2) 

where pk G K™, Pi = (diagand Q,\s a constant matrix. Here pj depends on pk continuously. The 
problem under consideration is to check stability of system ©, with matrices Pi satisfying the relation 
ll^ll < 1. The stability criterion using NL q approach says that if there exists diagonal positive definite 
matrices Dj such that WDjQjDJ^W < 1 for all j = 1,..., q(mod q ), then the system © is globally 
asymptotically stable. Using a suitable method I], the RNN defined in © can be transformed to form 
©■ Therefore the above criterion can be used to check stability of systems of form ©■ The NL q approach 
gives sufficient conditions for stability of nonlinear systems. However, there exist nonlinear stable systems, 
for which the NL q stability criterion is not satisfied. These nonlinear systems, for example, RNN, have 
shown promise in various applications [5] . 

Another stability criterion was developed using theory of absolute stability ( h, 0 , m, 0 - m )• a 
system to be analyzed for stability using this approach should be written in the automatic control form: 

x k+1 = Ax k + B4 ’ k , 

<r k = Cx k , (3) 

4>i = <t>i{°i), i = 1 • ■ • to, 

where, A, B, C are matrices of suitable size, i[) k = (ip 1 ,..., ip m ) is input vector at step k, a k = (a 1 ,..., <j m ) 
is the output vector at step fc, and {<Pi(')}kLi are nonlinear functions. 
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Before analyzing the stability of system m using theory of Absolute stability, it needs to be trans¬ 
formed to ©. State Space Extension method has been introduced [I] to transform RNN to ©• 

One of the significant contribution of theory of absolute stability is the frequency domain criterion 
( 0 , m M)- Frequency domain criterion gives necessary and sufficient condition for existence of 
quadratic Lyapunov function for class of systems © with functions </>(•) satisfying given local quadratic 
constraint. One of the most common constraints used for stability analysis of nonlinear systems is sector 
constraint, and the corresponding stability criterion is known as circle criterion. It has been shown in [T| 
that stability criterion given by NL q approach is weaker than the circle criterion. 

The circle criterion gives sufficient condition for stability of nonlinear systems, with nonlinear function 
</>(•) satisfying sector constraint. It only utilizes the fact that the nonlinear function </>(•) satisfies a 
given sector condition. It might happen that given a sector, defined by function </>(■), there exists a 
nonlinear function satisfying sector condition, such that the corresponding system is unstable. Additional 
information about the nonlinear function can be used to check stability of nonlinear systems of particular 
kind, for example RNN. A modified stability criterion using additional information about the nonlinear 
function,( e.g. monotonicity) has been developed in [T[. But this criterion has been shown to be essentially 
sufficient for systems with large number of nonlinear functions. In addition, this criterion is not applicable 
to some practically stable systems, for instance RNN. 

The stability criterion given by theory of absolute stability ( n, 0 , m, uni) checks necessary and 
sufficient conditions for existence of Lyapunov functions of a particular kind (e.g. quadratic forms). But 
there exists stable systems, for which quadratic Lyapunov functions do not exist. An alternative stability 
criterion has been proposed in j2j. 

Consider the system 

x k+1 = 4>{x k ) (4) 

Let Do denote the whole space of vector x k . Suppose there exists sets {D k } such that D k+ 1 c 
D k , <j>{D k ) C D k + 1 . If {D k } —> 0 (in Hausdorff metric), as k —» oo, then obviously system © is globally 
asymptotically stable. This approach is known as reduction of dissipativity domain. 

In order to implement this approach, the sets Dk need to be defined. A possible choice of D k is given 
by 

Dk- t-i — {x Dk ■ fk- t-i,j (*£) ^ 5 j — 1 ■ ■ ■ THk- (- 1 } 

where mk is the number of constraints at step k, f k ,j is a function, and a k +i,j = max l6 o t fkj(<p{x)). 
The set Dk is characterized by the set of pairs ( fk,j,®k,j ) where j £ {1... m}. A possible choice of fk,j{ ■) 
is linear functions. Then Dk takes the shape of a polytope. It has been shown (reference) that if system 
0 has a convex Lyapunov function, then there exists linear functions fk,j such that {D k } —> 0. 

The set D k is constructed by computing the value a k j for every j. Since the function is nonconcave 
over the set D kl it can have multiple points of local maxima. At every step k, the points of local maxima 
for the function /(</>(•)) need to be computed. 

Consider a single layer RNN with zero bias. Using substitution y = Wx, it can be expressed as 


Uk+i = W(f>(y k ) (5) 

For the case of RNN in ©,the function /(<(>(•)) is given by the inner product /( x) := (lj ,Wcj>(x)). 
We need to find points of local maxima for /(•) over polytopes defined by matrix of constraints, L = 
co/(Zi, I2, ■ ■ ■, lm) • It has been seen that, in all the cases, the function /(•) has points of local maxima on 
the boundary of the polytope. We will first locate the points of local maxima for /(•) on an arbitrary 
hyperplane. The subject of this paper is the solution to the following problem. 
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Problem Setting: Consider the hyperplane, P = {x : l T x = b} where l is a unit normal vector and 
b £ R. How many points of local maxima does the function f(x) = X^=i c i4 > ( x i)i d/0 for all i, have on 
P? Here </>(•) is standard neuron transfer function. 

This paper is organized as follows. In section 2, we will develop necessary and sufficient conditions 
for existence of points of local maxima. In section 3, some assumptions regarding function </>(•) will be 
listed. Section 4 gives the possible location of points of local maxima. Then we will talk about number 
of points of local maxima in main orthant and side orthant. We will conclude with the main result of 
this paper. 


2 Identify the points of local maxima 


We will develop the necessary and sufficient condition for a critical point to be a point of local 

r\ 2 r 

maximum for function /(•) over hyperplane P. Then D = \ X — X0 = diag(dj)" =1 is Hessian matrix, 

where dj = Cj(j)"{xj). Let K := (I — pj T )D(J — pj^) denote the projection matrix. 

Theorem 2.1. Suppose, Xg is a critical point of /(•) over P (i.e. I is parallel to V/(x g)). Then Xg is a 
point of local maximum of /(•) over P only if K <0. Moreover, if K has n—1 negative eigenvalues and 
one zero eigenvalue then xg is a point of local maximum. 


Proof. Necessity: Consider the Taylor expansion for /(•) in some neighborhood of xg. 


fix) = /(:e 0 ) 


d_l 

dx' 


"=XQ 1 £0 ) + 


X - Xq, 


dx 2 


=x 0 {x - X 0 ) ) + o(||x — £o|| 2 ) 


Since x € P, we have l T x = b and I t xq = b, hence x — Xg is orthogonal to l. Using the fact that, V/(xo) 
is parallel to l, we get V/(a; g) T [x — xq) = 0. 


Moreover since xg is a point of local maximum, we obtain ^ x — Xo, ^Jz\x=x 0 i x ~ x o)^ < 0 for all x 
such that l T {x — xo) = 0. Therefore 

y T Dy < 0 (6) 

for all y such that l T y = 0. 

Pick z £ R". Define y := (/ — j||jp)z. Then y £ P. Next we will show that K < 0. 

Pick 2 arbitrary. Then 


z T Kz = z T — 

T ( 




z 




z. 


Hence, z T Kz = [{I — pp)^ D^(7 — pp)- 2 ) = y T Dy. Using equation©, we have y T Dy < 0. 
Therefore, K < 0. 

Sufficiency: Suppose K has (n — 1) negative eigenvalues and one zero eigenvalue. This implies that 
z T Kz < 0 for all 2 £ K". We will show that if z £ {x : l T x = 0}, then z T Kz < 0. 

There exists orthonormal basis {vi,V 2 , ■ ■ ■ ,v n -i} of P consisting of eigenvectors of matrix K, with 
eigenvalues {Ai, A 2 ,..., A ra _i}. Since Vo = l is the eigenvector with zero eigenvalue, we get A i < 0 for all 
i £ {1... n — 1}. 
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Assume z € {x : l T x = 0}. Then z = Pj v j,Pj 6 R and, 

In —1 n —1 \ / n —1 n —1 

2 t a'z = / ^2p i v i ,'^2p i Kv i \ = (y^Piv^y^PiXjVi 
\i= 1 i-l / \i=l i—1 

n —1 n —1 

= 51 < rn ax(Xi) 55 Pi < °- 

2=1 2=1 

Pick x in neighborhood of xo on P. Then (xo—a;) T l. Put z = xo~x. We obtain (xq—x) t K(xo—x) < 0. 
Using definition of matrix K , we get {xq—x) t K{xq~x) = (xo—x) T D{xo—x) < 0. Since xq is critical point, 

(§^\x=x 0 ,x- x 0 } = 0. Using Taylor expansion for /(•), we obtain f(x) < f(x 0 ) in some neighborhood 
of xo on P. Therefore xo is a point of local maximum for /(•) on hyperplane P. □ 

In the following section, we will list some assumptions about the function </)(•). These assumptions 
will be used to show the main result of this paper. 


3 Assumptions about Cost function 


Notation : The following notation will be followed, unless specified. 

^'O) == Wq) ■= ■zp{i>(Pq)),hp(P,q j ,q n ) := qj, q n )),where ip(-), and h(-) are func¬ 

tions which will be defined later. 


Assumption 1: </)(■) € C 2 ,(p{—x) = —<p(x),(p'(x) > 0,x<p>"(x) < 0, for all x ^ 0, and lim a: _ ) . 00 <j>(x) < 
oo. x<j/'{x) < 0 implies that (f>'{x) is decreasing function for all x > 0. Hence exists. Denote 

0 , (-) _1 = ip(-). We get ip : (0, <//(0)] —> [0,oo). In addition, ip'(x) < 0 for all x. 

Assumption 2: x(ln \ip'(x)\)' is a monotonically increasing function of x. This implies that y) > 

0, where p > q. 


Assumption 3: Set h,(/3,qj,q n ) 


» t'Wln)' dp \_hp (0, qi,q n ) 


is sign definite, where qj < q n < qi. 


Assumption 4 : For all p > q, we have ^ ( ’t/jfpq ) ) < 0- 
Assumption 5: For all x > 0, we have — 0- 


4 Possible Locations of Points of Local Maxima 

In this section, we will use above assumptions to locate the possible locations of points of local maxima 
for function /(•) over hyperplane P. Recall f(x) = c i < ! ) { x i) an d P = {x : l T x = b}, where l = x 

is the normal vector. 

First, we change basis in order to get Cj > 0 for all j € {1... n}. Suppose Cj = 0 for some j € {1... n}, 
then the corresponding term in the sum is zero, and we obtain f(x) = Y^ik=i Cfc^fc- The problem is 
reduced to similar problem of dimension n — 1. Without loss of generality, we assume that Cj ^ 0 for 
all j € (1.. . n}. Next assume that Cjo < 0 for some j 0 € {1.. .n}. Using assumption (1), cp(-) is odd 
function. This implies that Cjo<p(xjo) = —Cjo(p(—Xjo). Hence, if Cjo < 0 then replacing Cjo by — Cjo, and 
ejo by —ej o, the function f(x) remains unchanged, and coefficient Cjo > 0. Without loss of generality, we 
assume Cj > 0 for all j £ {1... n}. 
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Next, we consider the signs of the components of vector l. If lj o = 0 for some j 0 G {1.. . n}, then 
Xjo can be increased arbitrarily, still ]Cj=i h x i remains unchanged. Hence, the function f(x) does not 
have a point of local maximum on P. Therefore we can assume that lj ^ 0 for all j G {1... n}. Next, 
assume that there exists jO, jl G {1.. .n} where jO ^ j 1, such that Ijo < 0 < l 3 \. Then we will increase 
Xjo and Xj\ such that ^" =1 IjXj is unchanged. It is easy to see that the function f{x) is increasing on 
P. The function f(x) does not have a point of local maximum on P. Hence, we can assume that for any 
j().jl G {l...n}, where j 0 ^ jl the product Ijolji is positive. Suppose lj < 0 for all j G {1 ...n}. 
Then we replace b by —b and vector l by —Without loss of generality, we assume that lj > 0 for all 
j G {1 • ••«}■ 

Theorem 4.1. Suppose Xq is a critical point for function /(•). Then, Xq is a point of local maximum only 
if the orthant has at most one negative coordinate (i.e. Xj < 0 for at most one j, where j G {1 ...n}). 


Proof. Notice that D = diag(d/)” =1 , and l denotes the normal vector. The characteristic polynomial of K 
is given by det(A I — K) = 0, where A denotes eigenvalue of matrix K. First we will compute det(A I — K). 


det(A I - K) = det (a I -(I- )D(I - |^)) 

= det (A / - D{I - 77 yrp)(/ - 77|7p 


det [\I — D^j det ^1 


l T (XI - D^Dl 


using Sylvester identity. 

Hence, 

det(AI — K) = det ^AI — Tjj 

= (n? =1 (A -di)) 

It follows from equation Q that A = 0 is an eigenvalue of K. Denote g( A) = J]”=i iu|| a '(A—rf ) • 

We can see that g{ A) has vertical asymptotes at dj. Suppose all dj ’s are distinct. Then, we can 
arrange them as di < d 2 < ... < d n . It is easy to see that for all dj > 0, lim, , ,+ g( A) = oo and 
lim. g( A) = — oo. If dj < 0, then lirri, , ,+ g(X) = — oo and lim, g( A) = oo. Since function g(-) is 

continuous in {dj,dj+ 1 ), there exists a root, A j of function g(-) in this open interval. Number A j is an 
eigenvalue of K for j = 1,..., n — 1. Thus, {0, Ai,..., A„_i} are all eigenvalues of matrix K. Now 
consider the general case. We order the values of dj: d\ < ■ ■ ■, d n . If dj < dj + 1 then the proof is same as 

above. If dj-\ ^ dj = dj +1 = ... dj+k ^ dj+k+u then there are k eigenvalues Xj = Aj+i = ... = Xj+k-i 
of K at point dj. Together with the zero eigenvalue the set of such numbers {Ay}, j G {1, 2,..., n — 1} 
is the set of all eigenvalues of I\. 

Claim 1 : dj > 0 for at most one j where j G {1 ■ • • n}. 

Suppose there exists dj , dk ,where j ^ k such that dk > dj > 0. Then g( A) = 0 for some A G (■ dj,dk )• 
This implies that matrix K has a positive eigenvalue, but K < 0 lTheorem l2.ll) . Hence, we arrive at a 
contradiction. Claim 1 proved. 

Case 1 : Suppose dj < 0 for all j. Using definition of dj, and assumption 1, we obtain Xj > 0 for all j. 

Case 2: Suppose dj > 0 for some j. It can be analyzed in a similar manner to Case 1. We obtain that 
Xj < 0 for some j. 


L + 


l r {XI - D^Dl 


£ 


A lj 


| 2 (A — di) ’ 


(7) 
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Combining the results of Case 1 and Case 2, we obtain that a stationary point, Xo, is a point of local 
maximum only if the orthant has at most one negative coordinate. □ 

Using Theorem 14.11 we can deduce that function f(x) can have points of local maxima in main orthant 
or side orthant with at most one coordinate negative. Next, we will show that f(x) has at most one 
point of local maximum in main orthant. 


5 Points of Local Maxima in Main Orthant 

In this section, we will show that f(x) has at most one point of local maximum in main orthant (i.e. 
Xj > 0 for all j G {1... n}). 

Proposition 5.1. The function f{x) = ]P’ l =1 c i ( t > ( x i) has at most one point of local maximum in the 
main orthant. 

Proof. This is obvious, since /(•) is concave over the main orthant. □ 

Next, we will present the necessary and sufficient condition for existence of point of local maximum 
in main orthant. 

Proposition 5.2. A point xq is a point of local maximum in main orthant if and only if there exists 
ft £ R + such that = where ft^- < cf'(0) for all j, and b is such that I'x o = b. 

Proof. Necessity: Suppose that Xq is a point of local maxima in the main orthant and Xq £ P, the 
hyperplane. Then x 0 is a critical point. We obtain V/( xq) = (31 for some /3 £ M. Since lj > 0 for all j, 
and = Cj(j)'(xj) > 0, we obtain that /3 £ R + . 

Hence, <p'(x = ft for all j £ {1.. . n}, which in turn implies that x^ = if(ftqj) where qj := , and 

if := (</>'(-)) _1 . Since xq lies on hyperplane P, we obtain b = I'x o = i f° r some [3 £ R + . 

Sufficiency: Assume that there exists f3 £ R + such that b = J2j =i Using definition of if{-), 

we get b = i ^( < ^ |/ )” 1 (/3y-)- Denote x J 0 := (</> , ) _1 (/3^ L ). This implies (f'(xP 0 ) = ft ^ for all j £ {1.. .n}. 

Therefore, f{x o) = fil. Hence Xq is a stationary point in main orthant. Since f{x) is concave over the 
main orthant, xq is a point of local maximum in the main orthant. □ 

In Theorem 14.11 we saw that a critical point xq for function f{x) = c i < / > ( ;r i) can be a point of 

local maximum only if it is lying in orthant with at most one negative component. In this section, we 
developed the necessary and sufficient conditions for existence of local maximum in main orthant. In 
addition, we also showed that f{x) can have at most one point of local maximum in main orthant. Next 
we will show similar result for the case of side orthant with one negative component. In later sections 
we will use side orthant to denote side orthant with one negative component. 


6 Points of local maxima in side orthant 

In this section, we will show that the function /(x), defined on the hyperplane, has at most one 
point of local maximum in side orthant. Here we have shown the result for the side orthant, with last 
component negative. Other cases can be analyzed similarly. 

First, we will develop the necessary and sufficient conditions for existence of local maximum in side 
orthant (i.e Xj > 0 for all j £ {1.. .n — l},x n < 0). Suppose xq is a critical point for function /(•) in 
the side orthant (i.e x J 0 > 0 for all j £ {1.. .n — 1},Xq < 0). Then, there exists ft £ R + such that 
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Cj(j)'(x J 0 ) = pij for all j G {1.. . 77 }. This implies x J 0 = sign(a^)(<//) -1 (/3^-) = sign(a;^)(^ , ) _1 (/3g J ) for all 
j G {1.. .n}. Recall that dj = Cj <j>' (xj). Using assumption 1, we obtain d\ < c ?2 < • • • < d n -\ < 0 < d n . 

Denote gi(fi) := I'x o = J2j=i — ln{4>')~ 1 (l3qn), where qj = jf?. Using the notation 

((i/) _1 := t/j, we obtain 


n — 1 

9l(P) = E ~ l n^(Pq n ) (8) 

7 = 1 

Theorem 6.1. Let g{ A) = Yfn=\ • If g'( 0) < 0 and gi(ft) = b, for some jf G R + , then x$ is 

a point of local maximum for the function /(•), over the hyperplane P. Moreover, xq is a point of local 
maximum only if g'( 0) < 0 and gi{/3) = b , for some ff G R + . 

Proof. Sufficiency: It is easy to see that lim A _^- g( A) = —oo and lim^_^ d + g( A) = — oo. We saw earlier 
that there exists unique A G (dj, dj+\) for all j < n — 2 and A is eigenvalue of K. Thus matrix K has 
n — 2 negative eigenvalues. 

So we have 2 roots of g( A) in the interval (d n -i,d n ). One of the roots is 0. Denote the other root as 
A. Notice that d n -\ < 0 < d n . This implies that A can be negative or positive. But under the assumption 
that g'( 0) < 0, we have A < 0. Moreover, A G (d„_i,0). Hence we conclude that matrix K has 77—1 
negative eigenvalues and a zero eigenvalue. Using theorem 12.11 xq is a point of local maximum. 

Necessity: Suppose that Xq is a point of local maximum. Then K < 0. This implies <7' (0) < 0 and 
b = - Li’iPqn) for some /3 G K + . □ 

In theorem IQ we developed the necessary and sufficient conditions for existence of point of local 
maximum in side orthant. These conditions can be rewritten in terms of function <?i(/3). To this end, we 
need the following equivalent statement. 

Lemma 6.1. Consider function g( A) as defined above. Then g'( 0) < 0 if and only if g\ (fi) > 0. 

Proof. Necessity: We can rewrite g'( 0) as 

n i2 1 n ;2 

o'(0) = V_ J— = — V_X_, 

d M\ 2 IKII 2 fo-0"(sign(a; J o )V>(^^)) 


where, dj = Cjfi” (xl) = Cjfi" (sign(a;^)^>(/3^-)), and Xq is critical point for /(•). Therefore, 


ff'(0) = 


1 ( \ ' l j9j 


1 / In.qn 

IRFVMftZn)) 


E 


7 = 1 


InQn 

fi" fif{fiqj)))' 


since </>"(•) is odd. 

Hence g'( 0) < 0 implies 

Qn^n qjlj 

Next, we want to express g[(/3) in a form similar to g'( 0). To this end, we need an auxiliary result. 
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Claim 2 : <f,"(2(pq)) = ^'(Pq)- Using definition of we obtain ip(cp'(x)) = x. Differentiating with 
respect to x we get ip'(cp'[x))(p" (x) = 1. This give us (p"(x) = ^pjppj^yj- Let (p'(x) = y. Then x = ip{y). 
Evaluating (p"(x ) we get (p"{x) = cp"{ip(y)). Hence = (p"(ip{y))- Claim 2 proved. 

From equation © it is easy to see that 

n — 1 

9i(P) = (/%) - lnq n ip’(Pqn) (10) 

3=1 

where, \ x =pq-= 4>'(x). Using Claim 2, equation ([101) can be rewritten as g[(P) = YZjZl vlffpqA) ~ 
V'lZlZq »)) ' Hence 5'(0) < 0 implies that g[(p) > 0. 

Sufficiency: It easily follows from claim 2, and equation m- □ 

As a consequence, we have the following result. 

Corollary 6.1. A critical point xq is a point of local maximum in side orthant if g[(/3) > 0, gi{/3) = b, 
and only if g[(l 3) > 0, gi{P) = b for some P £ M + . 

Proof. The proof easily follows from theorem 16.11 and lemma 16.11 □ 

Next we will show that the function /( x) = Xo=i c i4>{ x i) Las at most one point of local maximum in 
side orthant. Before going over the proof, we will prove some useful properties of function gi(-), which 
will be used frequently in following sections. Notice that 

n — 1 

9l{P) = ^2 l 3^(Plj) - l nf>{Pqn) 

3=1 

and, 

n — 1 

g[{P) = ^2 h9ji>'{pqj) - inq n ip'(Pqn) 

3=1 

Remark 6.1. If qj = q/~ for some j, k then the optimization problem can be reduced to similar problem 
of lower dimension. Therefore, in the future sections, we will assume that qj where j £ {1.. .n} takes 
distinct values. 

Lemma 6.2. Suppose that q n < qjo, where j 0 ^ n. Then the following statements are true. 

(i) g[(P) has at most two roots on the interval (0,/? mQa; ]. 

(ii) 9i{P) -t -oo, as p ->• p max • 


Proof. Denote h{P,qj,q n ) := and h(P,q h q n ) := where j,l £ {l...n- 1}. Recall that 

#m = %\s=to- 


_d_ 

9/3 


(i) Suppose q n < qj 0 for some jO £ {1...n - 1}. Let qi < q 2 < ... < qk < q n < qk+i <■■■ < q n - i- 
Using assumption 2, and definition of h (•, •, •), we get hp(p, qj,q n ) ^ 0. In assumption 3, we saw that 


hp(P,qj,<ln) 

hfilh,qi,qn) 


has the same sign for all ( P,qj,q n ,qi ) such that /3 £ (0,/3 max ], and 0 < qj < q n < qi. 


Using assumptions 2 and 3, we obtain 

d_(\ . I h P (P>qj,9n) : \ _ hp(P,qi,q n ) d r hp(P,qj,q n ) \ 
dp\° g hp{p,qi,q n ) ) hp{p,qj,q n ) d(3\hp(/3,qi,q n ))^ 
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Notice that for all pairs {qj,qi) such that qj < qi the expression hpp{P, qj,q n )hp{P, qi,q n )-hpp(p, qi, q n )hp(P, qj,q n ) 
is sign definite, where hpp(p,qj,q n ) = -^(hp(P,qj,q n )). This implies that 


k n— 1 


EE 

j= 1 l=k +1 



is positive 


hpp (P, qj,q n )hp{P,qi,q n ) - hp(ft, qj , q n )hpp(fi,qi,q n ) 


t^O, 


which, in turn implies that 


k n —1 k n —1 

^2,ajhp^{P,qj,q n ) ^ aihp(p,qi,q n ) -^2,oijhp(P,qj,q n ) ^ a t hpp(P, q u q n ) ^ 0 
j —1 Z=/c+l j=l Z=/c+l 


where, ay := Ijqj, and a; := hqi. 


Denote g{j.3) := 


Ey=i a jhp(P, qj, qn) 

Td=k+i a i h p(P’qi>qn) 


. We can see that left side of above equation is same as numer¬ 


ator of g'(P)- This implies that </(/ 3) is sign definite. Therefore there exists at most one /3, such that 

~\k 
J 3 


gift i) = -1. This implies that Xq= 1 ajhp(P, qj, q n ) + ELfc+i aih.f 3 (/3, qi,q n ) = 0 for at most one value of 


P- 


Using definition of hp(J3, q, q n ), we obtain a j-§p(KPi Qj>Qn)) + ELfc+ 1 a lWp( h (P> QhQn)) ~ l n q n 
has at most one root. Recalling definition of h(f 3, qj, q n ) we obtain that function X]j=i a j~§p ( - 0 ' [pg 1 \ ) + 
ElE+i ~ ^9" h as mos f one root. It is easy to see that above function is equal to 

( i ^{pq ) ) • Since ^(Pq) < 0j we obtain that g[(/3) has at most two roots in (0 ,/3 m ax\- This completes 


d 

d(3 


the proof for part(i). 


(ii) Using assumption 1, <j>' (x) is a decreasing function for all x > 0, and lim x _ ) . 00 (f>'{x) = 0 (since 
liniy^oo <fi(y) < oo). Since ip(-) := <(/(-) _1 , we get if : (0, <^'(0)] —> [0,oo). Moreover, < 0 for all 

x > 0. Using the fact that ip'(y) = 0"((qv)-Ri;)) ( by Claim 2 in lemma I6TT1) and </>"(0) = 0, we obtain 


ip'(y) -> -oo,as y -S> <^( 0 ). 


( 11 ) 


Recall that g[(P) = Y!j=i (j9jV , '(/%)-^9nV , 'C%0- Since, q n < q j0 = max(qj)^ =1 , we get qjoPmax = 
Hence using equation (fill) , we obtain that g\ (6) —> —oo, as ft — » Pmax- Proof for part (ii) completed. 

□ 

Remark 6.2. Similarly we can show that if q n = max(< 2 y)" =1 , then g[(/3) —> oo, as (3 P m ax■ 


Lemma 6.3. Suppose that q n > qj for all j G {1.. .n— 1}. Then, g[(p) has at most one root. Moreover, 
if lim/ 3 _>.o g[ (/?) > 0, then g[(p) > 0 for all ft G (0 ,p ma x\- 


Proof. Assume q n > qj for all j G {1... n — 1}. We will use contrapositive approach to prove this claim. 

ELi 1 ljqi i P' (Piqj) 

Suppose there exist Pi ^ P 2 such that g[{Pi) = g'i{p 2 ) = 0. Since g[{Pi) = 0 , we get — - -— 


1. Similarly, 


E?=i ljQji>'{P2qj) 


= 1. 


l n q n 4>'{Piqn) 

Denote := ctj > 0 for all j, and F(P) := EEi a j\ 


Lquip'{P2q n ) lnQn ' “ ±x J ' “““ " 1 3 Wi*)' 

Hence there exist Pi ^ P 2 such that F(Pi) = F(p 2 ) = 1. So there exist po such that F'(Po) = 0. This 

implies that Ej=i a o \&=Po = 0’ where, qj < q n for all j G {1.. .n— 1}. Since ctj > 0 for all 
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j £ {1... n — 1}, we conclude that (wjBTp)) \p=Pa i s n °t sign definite. Hence we obtain contradiction 

to assumption 2, which says that ^ ) > w h ere p ^ q is sign definite. Therefore, if > qj for all 

j £ {1.. .n — 1}, then g^i/3) has at most one root in the interval (0,/3 max ). 

Moreover, assume that lim^o g[ (/3) > 0, where g' 1 (/3) = YZjZi IjQj'&'WQj) ~ lnqnf>'(/3q n )- First, 
consider the case when lim^o g[{P) > 0. Then, the function gi(-) can either be identically increasing or 
gi(-) changes monotonicty. We will show that the second case cannot happen. Using the remark [6.21 we 
obtain 


9i (ft) ->ooas /3 -I /3 max- (12) 

Suppose by contradiction that gi(-) changes monotonicity. Using the fact limg^o g[(P) > 0, and (fl2l) . 
we get has two critical points, given by /3 and f3\. But this contradicts first part of this lemma. 

Hence g[(^) > 0 for all j5. £ (0,/3 max \. 

Next let lirng_»o 9\{P) = 0. Using remark RT7?1 and first part of this lemma, we obtain that g[(l3) > 0 for 
all /3 £ (0,/3 max]- d 

Remark 6.3. In lemmas \6.2\ and 1 6. 3[ we assumed that the last component is negative. Similar results 
hold true for side orthant with first or second component negative. These results will be used in the proof 
for case of two side orthants. 

Next, we will show the main result of the section. 

Proposition 6.1. The function f(x) = X)"=i has at most one point of local maximum in the side 

orthant. 

Proof. Recall that, gi(/3) = YJjZl ~ InifiPqn), and g[(/3) = J2jZ i - Lqntp'Wqn)- 

Case 1 : Suppose q n > qj for all j £ {1... n — 1}. We need to show that for a given b £ R, there exists 
at most one value of /3 £ (0 ,/3 m ax) such that gi((3) = 6, and g[(/3) > 0fcorollarv l6.ll) . Using remark 1(01 
we get that g[(/3) oo, as ft —> /3 m ax- Moreover, from lemma [6731 we obtain that that if q n > qj for all 
j £ {1...n — 1}, then g[(/3) has at most one root. Hence, there exists at most one value of /3 such that 
gi(/3) = b , and g[(/3) > 0. This completes the proof for Case 1. 

Case 2 : Suppose q n < qjo where j 0 ^ n. Using lemma [6771 we get that g[(0) has at most two roots in 
the interval (0 ,j3 m ax]- In addition, we showed in lemma [6~2l that g[(P) —oo, as (3 —> /3 m ax- Hence, for 

a given b £ R, there exists at most one /3 £ (0,/3 maa: ] such that gi(/3) = b , and g[(/3) > 0. Therefore f(x) 
has at most one point of local maximum in side orthant. Case 2 completed. □ 

In section 4, we saw that the function f(x) has at most one point of local maximum in main orthant 
or side orthant with one component negative. It might happen that there are two points of local maxima, 
one in main orthant and other in side orthant with one component negative. But in the next section we 
will show that f(x) does not have points of local maxima in both main orthant and side orthant with 
one negative component. 


7 Points of Local Maxima in Main and Side Orthant 


In this section we will show that the function /( x) = X)"=i c i4 > i x i)i defined on the hyperplane P, does 
not have points of local maxima in both main orthant and side orthant. In the main orthant f(x) is of 
the form 

n 

h{P) = ' 52 ki’(Pqj), 

l=l 
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and in side orthant it is of the form 


n— 1 

gi(P) = hi>(fiqi) - infPiPqn)- 

i =1 

We will first present two auxiliary results. These will be used to show that f(x) does not have points 
of local maxima in both main orthant and side orthant. 

Lemma 7.1. Suppose fi(/3) and gi(/3) are defined as above. Then the following are true: 

(i) fi(P) > 9i(P) for all fi G (0 ,/3 m ax] 

(ii) /((/ 3) < 0 for all ft G (0,/? max ]. 

Proof. It can be easily checked that 


fiiP) = ( 13 ) 

1=1 

and, 

n— 1 

9i(P) = ^2 ~ Lqni>\Pqn)- (14) 

1=1 

where iff) = (</>'(-))~\ ifffq) = \ s =f3 q ■ 

(i) We will first show that fi(ft) > gi(ft) for all ft G (0,/l max ]. Using assumption 1, we have q) > 0 
for all x. Hence if(x) > 0 for all x. Moreover, it is easy to see that the expressions for /i(/3) and 
gi{(3) are identical except the term l n if(l3q n ). Since l n f>{Pqn) > 0, we obtain /i(/3) > gi(ft) for all 
ft G (0, /3max\ • This completes the proof for (i). 

(ii) The proof follows from the fact that if'(x) < 0 for all x ( assumption 1 ). 

□ 


Now, we will present the main result of this section. 

Proposition 7.1. The function f(x) = ]U”=i Ci(j>{xi) has at most one point of local maximum in main 
orthant and side orthant. 


Proof. We need to show that f(x) does not have points of local maxima in both main orthant and side 
orthant. Using proposition 15.21 and corollary 16.11 we need to show that for given b G R, there does not 
exist pi 7 ^ fa G {t),ftmax} such that /i(/3i) = 6 , and g\{p 2 ) = fo, g'lifa) > 0. We will prove the above result 
using two cases. 

Case 1 : Let q n > qj for all j G {1.. .n — 1}. Using assumption 1, we get that iffy) < 0, and if(y) G 
[0, oo) where y G (0,0'(O)]. We obtain that lim^^o) if(y) = 0. Define y := ftq, then lim^^ if{Pq n ) = 
0, which in turn gives lim / 3 _ ) . j a maa . l n if(Pq n ) = 0. Hence 

fl(Pmax)) = gi(Pmax)- ( 15 ) 

Now we will look at following possibilities. 

Sub Case (i): Suppose lim^^o g\ (/3) < 0. First, we will consider the case when gi(/3) is decreasing 
identically. Using corollary 16. 11 f{x) does not have any point of local maximum in side orthant. So f(x) 
has at most one point of local maximum in the main orthant. 
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Next, suppose that gi{/3) changes monotonicity. We showed in lemma 16.31 that g[(P) has at most one 
root. Denote /3 £ [0,/3 moa; ] as the point where g i(/3) changes monotonicity. To summarize, fi(/3) and 
gi(/3) satisfy the following conditions : 

(i) f[{/3) < 0, and /i(/3) > g^fi) for all /3 £ (0,/3 m ax); see lemmaO 

(ii) g[(/3) < 0 on the interval (0,/?] and g[(0) > 0 on the interval [/3,/3 max]- 

(hi) fl(Pmax) = gi(Pmax)- 

Using the above conditions, we can deduce that for a given b £ R there does not exist /?i ^ fa such 
that fi(/3) = b and gi(/3) = b, g[(P) > 0. Hence, /(•) has at most one point of local maximum in either 
main or side ortliant. This completes the proof for Sub Case (i). 

Sub Case (ii): Suppose lim^o g[ (/3) > 0. From lemma RT~3l we get that gi(/3) is a monotonically 
increasing function. Using lemma 1770 /i(/3) > <7 i(/ 3) for all (3 £ [0 ,(3 ma x), and f[{P) < 0 for all f3. At the 
same time, gi{/3 max ) = fi(/3 ma x)- Hence, for a given b £ R, there exists at most one /3 £ (0 ,/3 max ) such 
that either /i(/3) = b , or gi(fi) = b,g[(p) > 0. 

Proof for Sub Case 2 completed. Combining the results of Sub Case 1 and Sub Case 2, we conclude 
that f{x) does not have points of local maxima in both main orthant and side orthant. This completes 
the proof for Case 1. 

Case 2 : Suppose q n < qjo , where jO £ {1... n— 1}. We will go over this case by contradiction. Suppose 
the function /( x) has two points of local maxima, one in each main and side orthant. This implies that 
there exist /3i ^ /3 2 , such that for a given b £ R, fi(fti) = <7i(^2) = b, and g[((3 2 ) > 0. Notice that /3i 
and fa lie in (0,/3 mQX ]. Recall that /i(/3) = E"=i (i^(/%)> and 9i(P) = E"=1 hWVj) ~ Ini’(fan)- 

Using part (ii) of lemma [6721 we get g[{(3) —> —00, as /3 —> /3 ma x- In addition, we assume that g' 1 (/3 2 ) > 0. 
This implies that there exists /3' £ (/? 2 ,/3 max) such that g[{P') = 0, and gi(/3') >b = /i(/3i) > fi{P ma x)i 
lemma mi . 

Since g[(p') = 0, we get l n = qri) E"=i {P' Qj)- This implies that ffi(/3') = E"=i l j (^iP'Qj)- 

•inifA q ^'(-P' q o)) • Moreover, we also have /i(/3) = E"=i h(piP9j) + q^pl^ ^iP'Qi))- Then ’ 
fi(Pmax) = E"=1 hi^iPmaxQj) + q ^{fiZ^q n ) Qj i>'(Pmax Qj ))■ Since gi{P') > fi(Pmax), we obtain 


h (I’iP'Qj) - > 55 l i (p(Pmaxqj) 


j= 1 


q n ip'{P'q n ) 


3 =1 


max Qn) 
Qnlf'iP max Qn) 


Qj V* (ftmax Qj ) ^ • 


We saw in section 4, that lj > 0 for all j £ {1... n}. Hence there exists j £ {1... n — 1} such that 


i’iP'qj) - \ Qj^’W'Qj) > P’WmaxQj) + Qn ^ qjtp'(,8 max qj )■ 


Qnf>'(P'q n ) 


Qn^’iP max Qn) 


We will show that the above inequality is not true. In other words, we will show that for all qj, where 
3 £ {!■••«- 1}, 


^ Qnl'iPln) ~ + qtl'ipZaxln) 


(16) 


Pick arbitrary qj , where j £ {1.. .n — 1}. Then we will consider the following possibilities. 

Sub Case (i): Let q n > qj. Using assumption 2, we have ,P > q is an increasing function of /?. 
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This implies > 0(since ijj(-) > 0). Notice that 


d_ 

d/3 









> 0. 


Qj ^'(ff'gj) 

gn Ip'iP'qn) 


q n ip'{ft q n ) 


The left hand side of inequality (fTBl) is an increasing function of /3. Hence, we obtain 


HP'Qj) 


-£&m u)- 

< HP max Qj) + 


HP max Qn) 
q n H{P max qn) 
HP max Qn) 
QnH{P max qn) 


QjHiP max Qj) 
QjHiP max Qj)- 


(17) 


since 


hp max Qn) 


■qjH(P max Qj) > 0. 


QnH(P max Q n) 

Therefore inequality (fl6|) is true. This completes the proof for Sub Case (i). 

Sub Case (ii): Suppose q n < qj. Using assumption 4 we have ( ^(flp) ) > where p > q. Then we 
obtain a sequence of inequalities, 


>fD>° 
( 


d/3 

d 

d/3 


1 




HPp) d P 

d 

HPp) < HPq)^t 


i P ) < 


HPq) d P 
{HPp)) 


( HPq)) 


since ^'(x) < 0: and V’( x ) > 0- 

Let p := qj,q := q n ,P := P’ , we obtain HP'Qj) ~ ) QjH\P'Qj) < 0- Hence left hand side of 

inequality (flbl) is negative. 

We have seen earlier that i//(x) > 0 for all x £ (0, <^(0)], and ^/(x) < 0 for all x £ (0, <j/( 0)). This implies 
that ipiPmaxqj) + ' qffl(Bnaxqn ) ^ ^ ( Pmaxqj) > 0. Therefore inequality (flbl) is true. This completes the 
proof for Sub Case (ii). 

Sub Case (iii): Assume that q n = qj. Then it is easy to see that V’(/3 , <b) — qtl>'(Fq n ) QjH(P' g j ) = 0- 
The left hand side of inequality (fTCl) is zero. The right hand side of inequality (flCH) is equal to 2 HPmaxqn)- 
If Pmaxqn = <t>'(0), then 4>(Pma X qn) = 0. Hence we get equality. 

Therefore, we obtain contradiction to our assumption. Combining the results of Case 1 and Case 2, 
we obtain that /(x) does not have points of local maxima in main orthant and side orthant. □ 

We have seen that the function f(x) = X)T=i c iH x i)-. Ci / 0 for all i £ {1... n} does not have points 
of local maxima in main and side orthant. Now, we will show similar result for two side orthants. 
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8 Points of local maxima in Two Side Orthants 


In this section, we will show that function f(x) = Y^i=\ c i4 > ( x i)i c i 7^ 0 for all n, does not have points 
of local maxima in two side orthants. We will show the proof for the case, when the first and second 
components are negative. Other cases can be analyzed similarly. 

In the first side orthant, f(x) takes the form 

n 

9i(P) = Y h^P^o) - h-tpiPqi), ( 18 ) 

1=2 

and in the second side orthant /( x) is of the form 

n 

92(13) = Y - hi’iPq 2)- (19) 

j=io¥2 

It can be easily checked that 

n 

9i(P) = Y (P q j) ~ l ^9ifp'(Pqi), ( 20 ) 

3 =2 

and 

n 

92(P)= Y h^'iPqj) - hq2^'{Pq2)- ( 21 ) 

i=i,i#2 

First, we will solve an auxiliary problem. The main result of this section will be an easy consequence 
of the solution to this auxiliary problem. 

8.1 Three Point Problem 

Lemma 8.1. Consider the functions g i(-) and g^f) defined above. Suppose fii, and @2 are critical points 
for gi(-) and 52O) respectively. Moreover assume that gi(/3i) — <72(/S2) < 0. Then the following inequality 
holds: 


P 2 r #(/32<?3) / 'fi(Piqi) _ if(Piq 2 ) \ 

2 Pi L-0/3(/5i9 3 ) 'fi’pifiiqi) ipp{Piq2)J 

-0/3(/?2g3) _ i’pifaqi) / ififcqi) _ \ 

ipp(Pm) fppWm) Wo^i) i>p(P2q2)' 

+ ^(A^) r ip(/3iqi) _ jKWi 
, >Pp((3iq2)\'ipp(l3iqi) ifpiPm) J 
#(/32g2) _ ifpifaqi) / ififcqi) _ \ 

MPi 92 ) ipp(P\qi)\ipp(p2qi) ippifcqs)' 

ifpi^qi) fjAJhq3)_ _ i>{Piq2) \ 
i>p(Piqi) 'i’piPm) ippiPrn)' 
fi(p2q2) _ \ ( ifpjfcqz) _ i’pjfcqi) 

fi>p{f32q2) Ipp(p2q3)' \il>p(Piq3) ippifiiqi) 
< 0 . 


( 22 ) 

(23) 

(24) 

(25) 

(26) 

fi’p(p2qi) _ ^8 (#292) _ ippjfcqz) _ ipp(P2q2) Y 
ipp(Piqi) i>p{Piq 2 ) ipp(Piq3) ippifim)!- 


Here, rpp(fiiqj) := -^(tpifiqj))\p=p t , and i G {1,2 },j G {1,2,3}. 
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Proof. Notice that the proof is quite technical. Since g[{0i) = 0, and 93(A) = 0, we obtain 

n 

hqiijj'(Piqi) = I2q24’ , (/3iq2) + ^2ljqj4>'(l3iqj), and 

3=3 


hq 2 ip' (A 92 ) = UgitAAgi) + ^ Ijqjtfifaqj) 

3=3 


Solving for 1 1 and Z 2 we get 


V{P\q2W{fogj) + ip'{P2q2)4’'{/3iqj) \ , , 
qi “ vV 7 '(/3i 91 )V' / (^ 2 92) - ^'(AgO^'CA®)/ ,an 

J—A 

/ VAAgi)A(Agj) + VAAgQVAAgjh 
92 ^ W , (A9i)^ , (A92) - if/ifaqiWWm)' 

j—a 


(27) 

(28) 


For simplicity, denote ^ , (A9 i)V’ , (A 92) — (fcqi)^'{Piq 2 ) := A. Since lj > 0 for all j, and ■!//(■) < 0, we 

get A > 0. 


Using equations (27D and (l28l) we can rewrite the expressions for gi(Pi) and <72(^2) as 


/a ^ [ ^(^ 192 ) (ip'ifaqiWiPiqj) + '0 , (/329j,)'0 / (/?i9i)A gA(Agi) {ip'(fcqj) + V ,/ (/^ 292 )V''(/5i9y)^ 

91(A) = 2^ (-A-j ft-V-A-J 

(29) 


3=3 

■I’iPiQj) 


92 

, and 


n V-, T V’(Agi) (ViPKbW (A&) + ’ t P' (^197)^(^292) 
92 (A) = ^ l 3 { -A- 

i=3 

+ ^(A9j) 

Since 91(A) - 92(A) < 0, we get 


9iA(A 92) / ^'(Agi)^'(A9j) + V ;/ (Agi)V ,/ (Agj) 

92 v A 

(30) 


91 (A) - 92 (A) = 


n 

&[-( 


V ,, (A92)V ,, (A9j) + V ,, (A92)V’ , (A9j)\/9jA(A9i) , 9jA(A9i) 


j=3 


)( ! 


9i 


9i 


+ 


V ,, (A9i)V’'(A9j) + V !, (A9i)V’ , (A9i)\/9ji/ ! (A92) . 9 jA(A92) 


92 


92 


W>(A9i) - V>(A9i)) 


< 0. 


Multiplying both sides of the inequality by 4AA9 i 92A we get 

n 

l i ~ ( 4 AA929jA'(A92)V ,, (A9j) + 4AA929jV’ , (A92)V |/ (A9j)) (AAgi) + V’(Agi)) 

3 =3 

(4AA9i9 J A , (A9i)V ?/ (A9j) + 4AA9i9 J A , (A9i)V ,, (A9i)) (V’(A92) + ^(A92)) 


■ 4AAgi92A(v>(A9j) - V'(Agi)) 


< 0 . 
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Since lj > 0 for all j, at least one of the coefficients in the above sum should be negative. Without 
loss of generality let j = 3, and we obtain 


4 P 1 P 29293 P'(0ig2)P'(P2g3) + O'(0292)0'(P 193 )) (p(0i9i) + 0(029i)) 

■ 4/3i/3 2 gig3 (p'(02gi)P'(Pi93) + P'^gsjP'tPigi)) (p(Pi92) + ^(^ 292 )) 


■ 4/3i/3 2 gig 2 A 


. 93 ) - 


< 0. 


We saw earlier that p'(x) < 0. Dividing both sides of above inequality by — 80fqiq2q3ip' (0iqi)ip' (0iq2)'4>' (0iq 3 ), 
we get 

1 #p2g2p'(P2g2) , 2/3 2g3 -i//(/3 2g3 ) N w UQ 

20igip'(0igi) v20ig2p'(Pig 2 ) 20iq 3 ip'(0 1 q 3 )) lQl 291 

_ l { 20 2 qiip'( 02 qi) , 2 02 q 3 ip'( 02 q 3 ) \ fJ , p \ , 1 m 

20 xq 2 ip' ( 0 iq 2 ) v2 / £?iQ , i-0 , (/3iqri) 20 1 q 3 tp , (0iq 3 )) 192 292 

_ 1 ^ 2/3 2 g 2 , i//(/3 2 g 2 ) 20 2 qi'il>'(0 2 qi) \,„ ua \ _ u o y> . n 

20iq 3 ip'(0iq 3 )\20iq 2 ip'(0iq 2 ) 2p 1 gip , (p 1 g 1 )) 193 293 


After distributing the terms on left side we obtain 


02 IY tM/3292) _ 

2/3? Ia p /3 (Pi 92 ) """ ippWiqa) 
( tpp( 02 qi) 0/3(02 93 ) 

W(/3i9i) 0/3(0 i93) 

fi>p(02q2) i>p{@2qi) 


ipp(02q 3 )\( P(0i9i) 


)( 


+ 


P/3(0i9i) 
_ P(0i92) 

0/3 (Pi 92 ) 

_ P(0i93) 

i)p(Pm) 

__ 0(0lg2) 

2/3i L0/3 (01 93 ) ^ 0/3 (01 9i) Ipp(0iq2, 

# 03292 ) / P(Pigi) i>(0iqz) ^ 


\it>p(0iq 2 ) ipp(0iqi) 

02 r#(^293) / P(0i9i) 


P(P2gi) \ 

i>p( 0 iqi)' 

0(#92) \ 

#(#92)2 

_##93)_y 

ipp(0iq 3 )J- 


ipp( 0 iq 2 ) \ipp( 0 iqi) 
ipp( 02 qi) r i>( 0 iq 3 ) 
ipp( 0 iqi) 'ipp( 0 m) 
#(# 92 ) ip( 02 qi) 


+ 


#(#92) 

#(#93) 

#(#93) 

il>p(02qi) 

tpp( 0 iqi) 


ipp( 0 iqi) 

0(0291) 

#(0i9i) 

0(0292) 


i>p( 0 iqs)' 

0(0i92) \ 

0/3 (0l92) ' 

_ 0/3(0292) _ 
0/3 (Oi 92) 
0/3(0293) 

0/3 (0l93) 
0/3 (029l) 


0(0293) 
0^3 (Oi 93 ) 
0(0292) 
0/3 (Oi 92) 

0(0293) ■ 


0/3(0l92) 0/3 (0l9l) 0^(0l93)- 


< 0. 


(31) 


We can see that the first three terms out of nine terms in the latter sum of inequality (1311) are monotonic 
in terms of 9. Next we will express the remaining six terms in similar form. We have 


0/3(0293) _ 0(p29l) _ P/3(P29l) 0/3(0292) _ 0(029l) _ 0^(02gl) 

0/3(0l92) 0/3 (Olgl) 0/3 (0291) 
0/3(0293) 0(0292) 0/3(0292) 


02 

20f Lp/3 (Pi 93 ) 1pp(0iqi) 0/3(029l) 
1pp(02qi) _ 0(0292) _ 0/3(0292) 

0/3 (Olgl) 0/3 (Pi 92 ) 0/3(0292) 

, 0/3(0292) 0(0293) 0/3(0293) 


0/3 (Pi 93) 0/3 (Ol 92 ) 0/3(0292) 
0/3 (p2gl) 0(0293) 0/3(0293) 


0/3(0l92) 0/3 (0l93) 0/3(0293) P/3(0l9l) 0/3(0l93 ) 0/3(0293)-! 
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For brevity, denote Xj = ^^(Piqj ) , and Vj = ^(p^q.) > wh ere J G {1,2,3}. In new notation, the above 
expression can be expressed as 


P 2 ( \ 

—2 I x 3 xiyi + x 2 xiyi - X\ x 2 y 2 - x 3 x 2 y 2 + x 2 x 3 y 3 - X\x 3 y 3 I . 

Zpi \ / 


(32) 


Adding and subtracting the terms, 4§zX 3 X\y 2 and 4§jX\X 2 y 3 to (ITil) . we obtain 

ZPi Zp 1 


( x 3 xiyi - x 3 xiy 2 + x 3 x x y 2 + x 2 x x y x - x 2 Xiy 3 + x 2 x x y 3 - x x x 2 y 2 - x 3 x 2 y 2 + x 3 x 2 y 3 - aq x 3 y^\ 

ZPi v / 


A 
2 0\ 


x 3 xi(yi - y 2 ) + x 2 x\{y\ - y 3 ) + (2/2 - y 3 ){x 3 x 1 - 2qa: 2 - 2:32:2)) • 


Using the definition of Xj, and yj, the above sum is same as 

02 [ ipp{02q 3 ) _ 4>p(02qi) / ip(02qi) _ ip(/3 2 q 2 ) \ j>p{P 2 q 2 ) _ j>p(02qi) / 4>(02qi) _ ip(02q 3 ) \ 

2^? (^ 193 ) ipp(0iqi)\ipp(02qi) ipp(02q2)' tpp{0iq 2 ) ipp(0iq x )\ipp{0 2 qi) ^p{02 qs)' 

/ V>(/3 2 g2) _ ip(02q 3 ) \ Z 'M&gs) _ #(/32gl) _ #(fegl) _ ^pU3 2 q 2 ) _ ipp{02q 3 ) _ #(^2g2)y 
^V’/s(^ 2 g 2 ) ^p(P 2 q 3 ))\ipp(Piq 3 ) ippifiiqi) ipp(0iqi) V’/ 3 (^ 192 ) ipp(Piq 3 ) ipp{0iq2)'- 

Combining the above expression with the first three monotonic terms of inequality (1311) . we obtain 


02 \ ^p{02q 3 ) / tpWiqi) 
2/3? L-0/3 (A 93 ) \ipp(0iqi) 
#(/32g2) / /Hffigi) _ 
ip1 3(^192) 'ippijhqi) 
#(/32gi) / ip(0iq 3 ) _ 
ippWiqi) W/3(/3ig3) 

_ ipp{ 02 q 3 ) _ #(/3 2 g 2 ) y 
ippifim) ipp{Piq2))- 


_ V , (/3ig2) \ ipp{02q 3 ) _ tppifcqi) / ipifcqi) _ v>(/3 2 g 2 ) \ 

ipp(0iq2)' ipp(Pm) #(/3igi) \ipp{0 2 qi) ipp (/3 2 g 2 )2 
^(/3ig3) \ i>p{02q2) _ ipp{0 2 qi) / ip(0 2 qi) _ ip(02q 3 ) \ 

i>p{Piq3)) ipp(0iq2) ipp(Piqi)\ipp(02qi) ipp{02q 3 )) 
^(/3ig2) \ / Ip(02q2) _ V>(/3 2 g 3 ) \ / #(/3 2 g 3 ) _ ippjPiqi) 

ipp{Piq2)' W/3(/3 2 g2) ^p(02q 3 )'^ipp{0iq 3 ) ipp(0iqi) 

< 0 . 


#(fegi) _ ipp(02q2) 
' i Pp{0iqi) #(/3ig2) 


□ 

Denote a:, = ^14^4,2/7 = , and 2 ? - = f . In the new notation, the above inequality can be 

2 tppiPiqj) 1 y 2 lkp{P2qj)^ 2 ^p(Piqj) ’ 1 2 

expressed as 



(33) 

Next, we will check whether there exist positive numbers /3i, /3 2 , gi, g 2 , and g3 such that inequality 
(1331) is satisfied. We will name this problem as Three point problem. This is the subject of discussion in 
the following section. 

8.2 Solution To Three Point Problem 

We will present the solution to three point problem using two different cases. In Case I, we will assume 
that g3 < max(gi,q 2 ). Without loss of generality we will assume that gi > g2- The other situation, where 
g2 > gi can be analyzed similarly. We will show that if gi = max(gj)? =1 , then (l33l) is not satisfied. To 
this end, we will present some auxiliary results. 
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Lemma 8.2. Suppose that 31 = max(gj)® =1 . Then, /3i > fc, where 3i) = 92 (^ 2 ) = 0. 


Proof. We saw earlier that A = tp'(Piqi)ip'(P2Q2) — ip' {P2qi)’f’ 1 {piq 2 ) > 0. This implies that 

ip'((3iq2) ip'{p2q2)' 


Suppose by contradiction, /3 1 < /? 2 - Moreover, assumption 2 gives ,p > q is an increasing function 
of /?. Using assumption 2 , and the assumption that > Pi, we get < - 777 ^^-• Here p = q±, 


and q = q 2 . This contradicts (1M1) . Hence, /3i > @2- 


^'(Pidz) ip'(p 2 q2)' 


□ 


Lemma 8 . 3 . Suppose that 31 > 33 > 32- Then the following inequalities are satisfied. 

(i) zi > z 3 > z 2 . 

(ii) yi > U3 > 2/2 ■ 

(iii) x 2 > X3 > X\ > 0 . 


Proof. Notice, Xj 


ippiPiij) ’ 


and Zj 


'l’(PiQj) 


(i) We will show that 21 > 22 • Remaining inequalities can be shown similarly. 

We need to show that ■ From assumption 4, we know that ^ ( v>(/ 3 g) ) < where 

p > q. This implies that ipp(/ 3 p)ip(/ 3 q) < ip([ 3 p)ipp{f 3 q). Put (3 = /3i,p = 31 , and 3 = q 2 . Then, we obtain 
tpp{piqi)tp{piq 2 ) < 4>{Piqi)tpp{Piq2), which in turn implies that Similarly, we can 

show other inequalities. Proof for (i) completed. 


(ii)The proof is identical to (i). 


(iii) We will show that x 2 > X 3 . Remaining inequalities can be shown similarly. 

We need to show that Using assumption 2 and the fact that /3i > P 2 , we obtain 

aKSififj) < °- Hence we obtain Siftr > Sifrtr- Moreover > since < °> we § et x 3 > ° for a11 

j € {1,2,3}. This completes the proof for (iii). 


□ 


Lemma 8.4. Suppose 31 > 32 > 33 - Then the following inequalities hold true. 

(i) £3(2/2 - y 3 ) < £2 - z 3 . 

(ii) 0 < X\ < x 2 < X 3 . 

(iii) Z!> z 2 > Z 3 . 

(iv) 2/i > 32 > 2 / 3 - 

Proof. Using assumption (2), we get > 0 ,P > 3- Recall that ip'(Pq) = f^ip{ s )\s=Pq- This 

implies that > 0, which, in turn implies > 0. Hence < 0, where 
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p > q. Here p = q -2 , and q = q 3. Since ip(-) is a non-negative function, we can rewrite the inequality 
_d_( b+£9)\ ^ n 
dfS ) < U aS 




lMff93) 

#(/3?2) 


ua ^ d (ifp{f 3 q 3 ) 


> 0 


The left hand side of the above inequality is the derivative of if(l 3 q 3 ) — il)^{fdq 3 ) ■ Since, /?i > / 3 2 , 

we obtain following sequence of inequalities 


ip(Piq3) - i>p{Piq3) 




> ^(^293) - ^(# 293 ) 


^(# 292 ) 

^/XA^)’ 


ippifhQa)- 

XXff 2g3) 

i>p(fiiqz) 


X(#292) _ 

#(# 292 ) 
i 2 ) 
{ippifaqi) 


_ ^($293) \ < ^(^192) _ 4 ’( Pi q 3 ) 

#(#293)' XX/X92) V’ / s(/ 3 i 93 )' 


This implies that 23(2/2^2/3) < ^2 —23. The proofs for remaining inequalities are similar to lemma 15751 □ 


Next, we will use the above auxiliary results to show that inequality ( 1551 ) is not satisfied. 


V’glfegj) _ tp{02qj) 
'I’plPiij)’I* 3 i’pifoqj) 


, and Zj = 


b( Piqj ) 

'l’p(Piqj )' 


Notice that 


Lemma 8.5. Suppose 91 = max(gj)^ =1 . Then inequality (1331) does not hold. 


Proof, (i) Suppose that 91 > 93 > 92 ■ We need to show that 

23(21 ~ + 2 ) + 2321(2/1 - 2/2) + 22(21 - 23) +2221(2/1 - 2/3) +21(23 - 2 2 ) + (2/3 - 2/2)(2122 +2 3 2 2 - 2321) 

is positive. From (i) and (ii) in lemma 18751 we obtain 

23(21 - 22) + 2321(2/1 - 2/2) + 22(21 - 23) + 2221(2/1 - 2/3) + 21(23 - 22) > 0 

It remains to show that (2/3 — 2/2X2122 + 2223 — 2321) > 0 . From (ii) and (iii) in lemma 1+51 we obtain 
2/3 > 2/2 and 21(22 — 23) >0 respectively. Hence 

23 (21 - 22) + 2 3 2i (2/1 - 2/2) + 22 (21 - 23) + 2 2 2i (2/1 - 2/3 ) + 2i (23 - 22) + (2/3 - 2/2)(2 i2 2 + 2 3 2 2 - 2 3 2i ) > 0 


(ii) Next suppose that 91 > 92 > 93- We need to show that 


23 (21 - 22) + 2321 (2/1 - 2/2) + 22 (21 - 23) + 2 2 21 (2/1 - 2/3) + 2l (23 - 22) + (2/3 - 2/2) (2122 + 2322 - 2321) ( 35 ) 


is positive. 


Using parts (ii) and (iii) of lemma l+Tl we get 22(21 — 23) > 0 and 21(23 — 22) < 0 . In addition, we also 
obtain that 


22(21 - 23) + 21(23 - 2 2 ) = 22(21 - 2 2 + 22 - 23) + 21(23 - 22) 
= 22(21 - 22) + (21 - 22)(23 - 2 2 ) > 0. 
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We can rewrite expression (l35l) as 

x 3 (zi - z 2 ) + x 3 x 1 (y 1 - y 2 ) + x 2 x 3 {yi - y 3 ) 

+ 0:2(2:i - z 2 ) + (oq - x 2 )(z 3 - z 2 ) + ( 2/3 - V2){xix 2 + x 3 x 2 - X 3 Xx) 

= x 3 (z 3 - z 2 ) +x 3 x 1 (yi - y 2 ) + x 3 x 2 (yi - y 2 ) 

+ %ix 2 (y 2 - y 3 ) + 0 : 2(21 - 22) + (a+ - x 2 )(z 3 - 2 2 ) + ( y 3 - y 2 ){xix 2 + x 3 x 2 - x 3 xi) 
= (-1 - z 2 )(x 2 + 2:3) + ( 2/1 - V2)(x 1 x 3 + XiX 2 ) 

+ (0:1 - 0:2) ((23 - 22) +X 3 (y 2 - 2/3)). 


Using parts (ii), (iii), and (iv) of lemma l8.4l we deduce that (zi—z 2 )(x 2 +x 3 )+(y\—y 2 )(xix 3 +xix 2 ) > 0. 
Next, it suffices to show that ^(23 — z 2 ) + x 3 (y 2 — y 3 )^j < 0. This is obvious from part (i) of lemma U~il 
Hence expression (1551) is positive. This completes the proof for part (ii). 

Summarizing the results from parts (i), and (ii) we obtain that when q 3 < q 2 ), the expression 

2:3(21 - 22) +x 3 xi(yi - y- 2 ) +0:2(21 - 23) + x 2 xi{yi - y 3 ) +2:1(23 - z 2 ) + (y 3 - y 2 )(xix 2 + x 3 x 2 - x 3 x 3 ) 
is positive. 

□ 


Next we will consider Case II. Assume that q 3 > max(qi,q 2 ). Without loss of generality assume that 
q± > q 2 . The other case can be analyzed similarly. We will show that 

x 3 (zi~z 2 )+x 1 x 3 (y 1 -y 2 )+x 2 (zi-z 3 )+x 2 xi(y 1 ~y 3 )+xi(z 3 -z 2 )+(y 3 -y 2 )(xix 2 +x 2 x 3 -xix 3 ) > 0. (36) 

Before going over the proof, we will present some auxiliary results. Notice that Xj = 1 Vj = 

V’lft Hi) ». _ Wig.,) 

VvX/ 3 29j)’ "2 't’p(PlQj)' 

Lemma 8.6. Suppose q 3 > qi > q 2 . Then the following inequalities hold true. 

(i) x 2 > Xi > x 3 . 

(ii) z 3 > zi > z 2 . 

(iii) y 3 > yi > y 2 . 

(iv) £1 > £2. 

Proof, (i) From assumption 2, we know that ^ > 0; where p > q. This condition can be restated 

as < °! where X 2 < Xl (lemma (E3)). This implies that > XlhTgi) ’ and we obtain 

2:2 > aq. Here ij>p(J3p) := ^(if(Pp))- 

Similarly, we can show that x\ > X 3 . Hence we get £2 > > £3 > 0. 


(ii) Using assumption 4, we know that ^ ( vi(/ 3 g) ) < ^ f° r P > V- This implies that > 

tXXXpMX?)- If we put X = Xi ,P = 9i, and q = q 2 , we obtain ■ This implies z 3 > z 2 . 

Similarly, we get 23 > 21. Hence z 3 > z\ > z 2 . 


(iii)The proof is similar to (ii). 


("I We need to show that 

#(•) < 0 ) 


This is equivalent to showing that -(since 
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Using assumption (2), we obtain > 0 . This implies ^gg^j > lim^^ gg ■ Now we 

will show that lim / 3 2 _ ! . 0 + exists. 

Since > 0 , we obtain that is a decreasing function of / 3 , as f3 —>■ 0 + . Hence 

{ ^g^j } has an upper bound for small but positive values of /3. Moreover since ipp(‘) is negative, { ^g^j } 
has a lower bound too. We obtain a monotonic and bounded sequence, hence lim^ 2 _ > ,o+ exists. 

Using L’hospital’s rule we get lim /32 _ M3+ ^g^^j = lim /32 ^. 0+ . Notice that ip(y) -> oo, as y -)> 

0 +. In addition, assumption (4) gives < 0 ,p > q. This implies that lim /32 _ ! . 0 + ^g^j > ^g^g - 

Hence we obtain the following chain of inequalities 

i’pifaqi) lim ^pifaqi) lh lim ipjfcqi) V>(/3igi) 

V’sC^tfe) / 3 2 — 5 - 0 + il>p{l32q2) / 3 2 + 0 + ip(/3 2 q 2 ) 4>(/3i 92) 

Therefore we get ^ggjg^j < ^ggg^j ■ Proof for (iv) completed. 

□ 

Lemma 8.7. Suppose that <73 > qi > q 2 - Then inequality holds true. 

Proof. From lemma [H~2l we obtain that fd\ > ft 2 . The left hand side of ( 1361 ) can be rewritten as 

£3(21 - Z 2 ) + XlXz(yi - 2/2) + £2,21 - £223 
+ £ 1(23 - z 2 ) + x\X2{yi - y2 + y 2 - 2/3) 

+ (2/3 - 2 / 2 )£l £2 + (2/3 - y 2 )(X2X3 - £i£ 3 ) 

= £3(21 - 22) + Xxx 3 (yi - y 2 ) + 23(xi - £ 2 ) 

+ X2Z1 - £122 + £l£ 2 ( 2/1 - 2 / 2 ) + ( 2/3 - y 2 )x 3 (X2 - £l). 

Using parts (i), (ii), and (iii), of lemma [8T6l we get £3(21 — 22) > 0,£i£3(?/i — y 2 ) > 0, 23(£i — £2) > 0 , 
and (2/3 - 2 / 2 )£ 3(£2 - £1) > 0 . 

Hence £3(21 - z 2 ) + x 3 x 3 fyi - 2/2) + 2 3 (£i - x 2 ) + x 2 Zi - £12 2 + £i£2(2/i - 2/2) + (2/3 - y2)x 3 {x 2 - £1) > 
x 2 z\ — X\z 2 + {xix 2 ){yi — y 2 ). Next we will show that x 2 Z\ — X\Z 2 + (xiX2)(yi — 2/2) > 0 . Since 
£i£2(2/1 — 2/2) > 0 (using(iii)), it suffices to show that x 2 z\ — X\z 2 > 0 . This is equivalent to showing that 
^2 > ^2. But we saw in (iv) that > 0 . Hence we obtain 

£3(21 - 2 2 ) + £l£3(2/1 - 2/2) + £221 - £ 2 2 3 
+ £1(23 - 2 2 ) + £l£2(2/1 - 2/2 + 2/2 - 2/3) 

+ ( 2/3 - 2 / 2 )£l £2 + (2/3 - y 2 ){x 2 X 3 - £i£ 3 ) > 0 . 

This completes the proof for the case when q 3 > qi > q 2 - □ 

This completes the solution to Three Point problem. Now we have all the required tools to present 
the main result of this section. We will show that the function f{x) = ^"=1 c i</>(£i),Ci 7^ 0 for all i has 
at most one point of local maxima in two side orthants. 


8.3 Number of Points of Local maxima in two Side orthants 

Recall that in the first side ortliant f(x ) is of the form g i(/3) = IjiPiPQj) ~ h4’((dqi), and in the 
second side orthant f{x) is of the form g 2 {/d) = 1 ^2 Ij'&iPQj) ~ hi>(Pq 2 )- It can be easily checked 
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that g[(/3) = ]U” =2 hnVifaj) ~ hqii>'(/3qi), and g' 2 (P) = ^= 1,^2 ~ hq 2 V(Pq-i)- 

We will first show that for small values of /?, either gi(/3) or g 2 (/3) is a non-increasing function. Since 

ip'(x) < 0, we obtain that sign(#i(/3)) = -sign^"= 2 l j q j t'(hqi) ~ liqi )' Sim ilarly for g 2 {fi). 

Suppose liing-m 9i {P) > 0, and lim /3 _ > o g ' 2 (/3) > 0. This implies that ( I 2 j =2 t'lhqi) ~ llQl ) < 0 

for values of beta close to zero. Since lj, and qj are positive for all j, we get l\q\ > Z 2 g 2 ■ Similarly, 

lim^^o5 2 (/3) > 0 implies that l 2 q 2 > hqi^jpjj^- From above inequalities we obtain and, 

r 51 < Hence contradiction. 

<292 V (p?l) 

Proposition 8.1. The function /( x) = Y^=i c i4 > i x i) does not have points of local maxima in both side 
orthants. 

Proof. We showed earlier that for small values of /3, either gi((3) or g 2 (/.3) is non-increasing function. 
In this proof, we will only consider the situation, when lim^-,. 0 g'i(P) < 0- Other case can be analyzed 
similarly. 

Case I : Suppose qi > qj for all j 6 {2... n}. Then g[{(3) —> oo, and g 2 (/3) —> — oo, as /? —> f3 m ax■ From 
remark [6.31 we obtain that 5 i(-) has at most one stationary point, and g 2 (-) has at most two stationary 
points. These observations will be used throughout the proof of this case. We will consider the following 
sub cases. 

Sub Case (i): Suppose g 2 (/3) < 0 for all /?, and lirng-m g [(/?) < 0. Using the fact g[(j3) —> oo, as 
/3 (3max , we deduce that gi(/3) changes monotonicity exactly once. So for a given b € R, there exists 
at most one f3\ £ [0 , (3 m ax\ such that gi((3i) = b , and g[(j3 1 ) > 0. Using corollary 16.II we conclude that, 
f{x) has at most one point of local maximum in the side orthant with first component negative. 

In addition, g' 2 (/3) < 0 for all /3 implies that the function f(x) does not have any point of local 
maximum in the orthant with second component negative ( corollary 16.111 . Hence f(x) has at most one 
point of local maximum in the side orthants. This completes the proof for (i). 

Sub Case (ii): Suppose lim^o g[((3) < 0, and lim^o g 2 (/3) < 0. We know that the function gi(/3) has 
at most one critical point in the interval (0, /3 m ax}- Together with the fact that g[(l3) —> oo, as j3 —> /3 max , 
we deduce that g\(/3) changes monotonicity once. Denote critical point of gi{/3) as /3i. It can be easily 
checked that gi{/3) is increasing on the interval (/3i, Pmax], and decreasing otherwise. 

Meanwhile, since g 2 (/ 8 ) —> —oo, as /? —> (Umax, and liing^o g 2 {P) < 0, we conclude that g 2 {/3) is either 
monotonically decreasing or changes monotonicity. If g 2 (/3) is monotonically decreasing, then we get the 
situation similar to Sub Case (i). Here, we will consider the situation when g 2 (/3) changes monotonicity. 
It is easy to see that function g 2 (l 3) has two critical points, given by /3", and /? 2 . It can be easily checked 
that g' 2 (/3) > 0 for all f3 £ (/3",/3 2 ), and g 2 (/3) < 0 elsewhere. 

Suppose by contradiction, f(x) has two points of local maxima, one in each orthant. Hence there 
exist /?' ^ j3’" such that for a given b £ R, we have gi(/3') = g 2 {/ 3 "') = b, gi (/?') > 0, and <7 2 (/3"') > 0. 
Here /?' <E (/3i,/3 m ax), and /3"' <E (/?",/3 2 ). 

Since gi{-) is increasing on the interval (/3i, /3 m ax), and /3' £ {/3i,/3 max ), we get gi((3i) < gi(/3') = b. 
Similarly, we obtain g 2 (/3 2 ) > b. This implies that gi{Pi) — g 2 (/3 2 ) < 0. This leads us to the Three point 
problem. See Lemma (18.ID . Hence we get 

X3(zi-Z2) + X 1 X3(yi-y2) + X2{zi-Z3) + X 2 Xi(yi-y 3 ) + Xi(Z3-Z2) + (,y3-y2)((xiX2 + X2X3-XlX3) < 0. 

But in Lemma T8.51 we showed that if q± = max(gj)j =1 , then 
x 3 (zi - Z 2 ) + XiX 3 (yi - y 2 ) + x 2 (zi - Z 3 ) + X 2 Xi(yi - y 3 ) + xi(z 3 - Z 2 ) + (y 3 ~ U 2 )((xiX 2 + X 2 X 3 - X 1 X 3 ) > 0. 

Hence contradiction to our assumption. Therefore, f(x) cannot have points of local maxima in both 
side orthants. Proof for (ii) completed. 
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Sub Case (iii): Suppose lim^^o 9i(P) < 0, and lim^o g' 2 (P) > 0. We saw earlier that g((p) oo, as 
P —> Pmax- Since g((P) has at most one root, there exists unique Pi G (0,/3 max ) such that g((P i) = 0. It 
can be easily checked that g((P) < 0 in the interval (0,/3i) and g((P) > 0 in the interval (Pi, Pmax}- 

At the same time, g 2 (P) —> —oo, as p —> Pmax- Since g 2 (P) has at most two roots, there exists unique 
P 2 G (0 ,pmax) such that g 2 (fj 2 ) = 0. It is easy to see that g' 2 (P) > 0 in the interval (0 ,p 2 ) and g' 2 (P) < 0 
in the interval (p2,Pmax)- 

Assume that f(x) has two points of local maxima, one in each side orthant. Using corollary 16.11 we 
conclude that for some b G R, there exists /?' / /3 such that gi(P') = g 2 (P") = b > 0 ,g'i(P') > 0, and 
g' 2 (P") > 0. It can be easily checked that P' G (Pi, Pmax), and ft" G (0,P 2 )- 

Notice that gi(Pi) < b and g 2 (p2) > b. Hence gi(Pi) — < 72 ($ 2 ) < 0. This case can be analyzed in similar 
manner as Sub Case (ii). We will obtain a contradiction to the fact that gi(Pi) — <72 (A 2 ) < 0. Hence f(x) 
has at most one point of local maximum in both side ortliants. This completes the proof for (iii). 

To recapitulate, when qi > qj for all j G {2.. . 71 }, the function f(x) has at most one point of local 
maximum in two side ortliants. This completes the proof for Case I. 

Case II: Let q 2 = max(< 7 j)" =1 . The proof for this case is similar to Case I. 

Case III. Let qjo = max((/j)" =1 . Here j 0 7 ^ 1 , 2. Without loss of generality, let j 0 = 3. Then g((P) —> 
— 00 , and g' 2 (P) —> — 00 , as p —> pmax■ Moreover, using remark l6~3l we obtain that both functions < 7 i(-) 
and g 2 (-) have at most two stationary points. These observations will be used throughout the proof of 
this case. We will consider the following sub cases. 

Sub Case (i): Suppose g((P) < 0, and g' 2 (P) < 0 for all values of p G (0,/3 max ). Using corollary 16.11 
f(x) does not have any point of local maximum in both side orthants. 

Sub Case (ii): Assume that lirqg-s-o g'\(P) < 0 , and g 2 (p) < 0 for all P G ( 0 ,Pmax)- If g'i(P) is negative 
for all P, then we obtain same situation as Sub Case (i) above. 

Here, we will assume that that gi(P) changes monotonicity. We know that, gi(P) can have at most two 
critical points in ( 0 ,p max )- Moreover, g[(P) —> — 00 , as p —> Pmax■ Hence there exist P' ^ P" such that 
g'i(P') = g'\(P") = 0. Notice that gi(p) is increasing on the interval (/3 , ,/3") and is decreasing elsewhere. 
This implies that for given b G R there exists at most one value of /3, say P 3 such that giiPs) = b , and 
g[(p 3 ) > 0. It is easy to see that P$ G ( P',P"). 

Next, g' 2 (P) < 0 for all p G ( 0 ,p max ) implies that f(x) does not have any point of local maxima in 
the side orthant with second component negative ('corollary 16. ID . 

From the above discussion, we conclude that /( x) has at most one point of local maximum in side 
orthant with first component negative. This completes the proof for (ii). 

Sub Case (iii): Assume that lim^o g'\(P) < 0, and Ymip^o g' 2 (P) > 0. Since g 2 (P) —> —00 as p —> 
Pmax , we deduce that either <72 (/?) has one stationary point or three stationary points. But we know that 
g 2 (P) can have at most two stationary points. Therefore there exists p 2 G (0, Pmax) such that g' 2 (P 2 ) = 0. 
It can be easily checked that g 2 (P) > 0 in (0 , p 2 ) and g 2 (p) < 0 in ( P 2 ,P max )• 

In the side orthant with first component negative, we can have two possibilities, either g((p) < 0 for 
all p G ( 0 ,pmax), or gi(p) changes monotonicity. 

If g'i(P) < 0 for all p G ( 0 , Pmax), then f(x) does not have any point of local maximum in side orthant 
with first component negative ( corollary 16. ID . Moreover, since g 2 (P) has exactly one critical point, there 
exists at most one value of p G ( 0 , ^2) such that for given b G M, <72(/?) = b, and g 2 (P) > 0 . This implies 
that f(x) has at most one point of local maximum in side orthant with second component negative. 

Next, suppose g i(-) changes monotonicity. Since g[(P) —► — 00 , as p —> p max , the function gi(p) will 
have exactly two stationary points. Denote the two stationary points, as Pi, and P", where Pi < p". 
Notice that g((P) > 0 in the interval (Pi,P") and is negative elsewhere. In addition, g 2 (P) > 0 in (0 ,p 2 ). 

Suppose f(x) has two points of local maxima, one in each side orthant. This implies there exist 
P 3 7 ^ P4 such that for a given b G M, we have gi(P 3 ) = g2(Pi) = b > 0 ,g'i(P 3 ) > 0, and g 2 (P4) > 0. It is 
easy to see that P4 G (0,^2) and p 3 G (Pi,P"). 

Since g((P) > 0 for all P G (Pi,P") and gi(p 3 ) = b, we get gi(Pi) < b. Similarly we obtain g 3 (p2) > b. 
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Hence we obtain gi(/3i) — g 2 (/3 2 ) < 0. This leads to the Three Point problem, which in turn implies that 
2:3(21 - 22) +xixz{yi - 2/2) +2:2(21 - z 3 )+x 2 xi(y 1 - y 3 )+xi(z 3 - z 2 ) + ( y 3 - y 2 )((x!x 2 +x 2 x 3 - 2:12:3) < 0. 

But in lemma [+71 we showed that if q 3 = ma x(qj)^ =1 , then 

x 3 (z 1 -z 2 )+x 1 x 3 (y 1 -y 2 )+x 2 (zi-z 3 ) + x 2 x 1 (y 1 -y 3 )+x 1 (z 3 -z 2 ) + (y 3 -y 2 )((x 1 x 2 +x 2 x 3 -x 1 x 3 ) > 0. 

Hence contradiction to our assumption. This completes the proof for Case III. Therefore, f(x) does 
not have points of local maxima in both side orthants. 

□ 


9 Special Case 

In previous sections, we have analyzed the behavior of function f(x) = )C"=i Ci^ixi) only in the open 
orthant (i.e. Xj 7 ^ 0 for all j £ {1... n}). Moreover, we also noticed that f(x) can have at most one point 
of local maximum in the open orthant on hyperplane P. But it might happen that function /(•) has two 
points of local maxima, one in the open orthant and other on the boundary of the orthant (i.e. Xj = 0 
for at least one j £ {1... n}). In this section, we will show that this cannot happen. 

First, notice that the function /( x) cannot have infinitely many points of local maxima on the bound¬ 
ary of the orthant. Suppose /( x) has infinitely many points of local maxima, {xi}'^Z 1 . Since Xi lies on 
the boundary, then for all i, we obtain xj = 0 for at least one j £ { 1... n}. This implies that g(/3k) = b, 
for the sequence where g((3) := YPj =1 Recall that ip(-) = </f>'(-) _1 . Using the fact that 

(j>{-) is an analytic function, we obtain that g(/3) = b. This implies that g(-) is constant function. Hence 
contradiction. Therefore, the points of local maxima on the boundary of the orthant are isolated. 

Lemma 9.1. Suppose x 3 is a isolated point of local maximum for function /(•) on hyperplane P = 
{ 2 : : l T x = b}. Then, for all e > 0 there exists Sq > 0 ,l,b such that ||^ — Z|| < <5o, \b — b\ < S 3 , and a 
point Xq with all components nonzero such that Xq is a point of local maximum of function /(•) over 
P = {x : lx = b} C\ || 2 "o — aToII < £■ 

Proof. Since 2:0 is a isolated point of local maximum for /(•) on P, there exists sufficiently small £q > 0, 
such that for all x, where x £ P and || 2 : — 2 : 0 1| < £ 0 , we get f{x) < f{x 0 ). 

Fix a positive number e < e 3 . Since x 3 is a point of local maximum, there exists /3 £ R + such that 
x J 0 = ±i/j((3P-) for all j £ {1.. .n}. If all components of 2:0 are nonzero, then we are done. Suppose that 
2: 0 has zero components. Without loss of generality, let = ... = x™ = 0. Then, using assumption (1), 
we obtain /3 = 4>'(0) for all k £ {1 ... in}. 

Since /(•) is continuous function, there exists <5o > 0 such that /(•) has a point of local maximum 
over the set {x : lx = b} f) {x : ||x — II < £} for all vectors l, and numbers b such that ||Z — I|| < <5o, and 
\b-b\<S 0 . 

Fix positive numbers 6 , <5i, such that <5i < 6. Denote l\ = l\ — 5\, lj = lj—S , for 2 < j < m, lj = lj for j > m. 
Denote by Jo the point with coordinates aio 1 = ±V ) (/ ^ 1 ~ < 51 ); xq-* = for 2 < j,m ,xrf = xl for 

Cl Cj 

j > m. Denote b := l T £ q. Assume that ^- < y- for all j £ {2... 71 }. 

We will choose 6 such that ||Z — /|| < 5q , and \b — b\ < S 0 . Then /(•) has a point of local maximum, x 
over the set P = {x : l T x = b} fl {x : ||a; — 2 : 0 II < £}• There exists /3 > 0 such that x J = ±if0—) for all 

C 3 

j £ {1... n}. If all components of x are nonzero, then we are done. 

Assume at least one component of x is zero. Due to choice of Si, it should be the first component x 1 . 
This implies that jdjf = and we get lx = b. Next consider positive number 5 and a vector l with 
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components li = l\, lj = lj — S for j £ {2... n}. Define the vector x with components & = ±^(/3^-), 
where j £ {1.. . n}. The number <5 is chosen in such way that \\l — l\\ < 6qJ t x ^ b, and > ff: for 
j £ {2.. .n}. Consider the hyperplane P = {x : l T x = b}. There exists a point of local maximum, y of 
function over P fl \\y — cco|| < £■ Therefore there exists /3 £ R+ such that = ±.if0^-) for all j = 1... n. 
Since l T x ^ b, we get that x ^ P. 

If all components of y are nonzero, then we are done. Assume y has a zero component. Then due to 
choice of <5i, <5, we get y 1 = 0 ^ 0 wherej £ {2.. .n}. This implies that ^ = Sh. = <^/(0). Therefore 

/3 = /?. But in this case y = x, and therefore vector y does not belong to hyperplane P. Hence we obtain 
contradiction. Therefore components of vector y are nonzero. Thus y is a point of local maximum of 
function / over P such that ||! — Z|| < <5o, \b — b\ < 6 q, and \\y — *o|| < £• 

□ 


As a consequence we obtain the following result. 

Corollary 9.1. The function fix) does not have two points of local maximum; one in open orthant and 
other on the boundary of orthant. 

Proof. Suppose Xo , and x\ are two points of local maxima for function /(•). Without loss of generality 
we can assume that xo is in the open orthant i.e. ij / 0 for all j £ {1 .. .n}. Since /(•) has at most 
one point of local maximum in the open orthant, Xo is isolated point. Then using lemma liPl we can find 
another point of local maximum Xq such that all the components of xq are nonzero. 

Using similar reasoning as above we can also find another point of local maximum X\ such that all 
the components of x\ are nonzero. Then we end up with two points of local maxima in the open orthant, 
on the new hyperplane P. Hence contradiction. □ 


10 Main Result 

In Section 3, some assumptions about the function (/>(■) were listed. Those assumptions turned out to be 
sufficient conditions for the above results to be true. Therefore, we obtain the following result. 

Theorem 10.1. Consider the function f{x) = Cif'ixi), where c; 0. Suppose the function <£>(■) 

satisfies the conditions (1) — (5) stated in Section 3. The function f(x) has at most one point of local 
maximum on the hyperplane, P defined by {x : I'x = b} for some b £ R. 

Proof. The proof follows from the previous results. □ 

In the next section, we will present couple of examples of neuron transfer functions, </>(•), which satisfy 
the conditions proposed in section 3. 


11 Examples 

In section 3, some assumptions regarding function were presented. In the previous section, we saw 
that these assumptions turned out to be sufficient conditions for Theorem [THU] to be true. In this section 
we will check these properties for two functions, namely tanh(-) , and arctan(-). 


1. <j){-) £ C 2 ,(f(—x) = —cf){x),(j)'ix) > 0, x<p"(x) < 0, for all x 0, and lim^^oo (f>{x) < oo. 
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(i) Suppose (/>(■) = tanh(-). Notice that (/J'(x) = — ^ ^ i s eas Y to see t ^ ie the g ra ph of 

cp "(•) lies in second and fourth quadrants. Remaining properties can be easily checked. 

(ii) Now we will consider the case when </>(•) = arctan(-). Notice that = — jj^sys ■ Remaining 
properties can be checked easily. 

2. The function <p'(-) is invertible. 


(i) Suppose that cp(x) = tanh(x). Then cp'(x) = seclr 2 (a;), where x > 0. Denote sech 2 (a;) = y. This 
implies ip(x) = arctanh(Vl — x), where ip(-) := 


(ii) Next, consider <p(x) = arctan(aj). This implies <p'(x) = —Hence ip(x) = y — 1. 

3. ip(x) is decreasing function of x. 


(i) Suppose ip(x) = arctanh(v / l — x). Using definition of arctanh(a;), we get arctanlr(\/l — x) 

— In f _ n ow , we w m evaluate ip'(x). 

2 VI — — x' 


' 4 ' w =&( 


1 f-l + y/T^ l + v / T TT 


2Vl~. 


2Vl~- 


2xy/l — X 

(ii) Similarly, for the case of ip(x) = y — 1, we obtain ip'(x) 

4. x(ln \ ij/{x)\y is a monotonically increasing function of x. 


2xiy/T^. 


< 0. 


(i) First, consider ip(x) = arctanh(\/l — x). Then 
(in \ip' (x)\y = 1 


2xy/l — x 
3x — 2 




—21 VI — x — 


2x{\ — x) 

d , ,, , ,,, ,1/3 — 3a; + 3x — 2 n 


This implies that — (x(ln \ip'(x)\y) = 

dx 2 V 


2^1^. 


1 


(1 — x) 2 ) 2(1 — x) 2 

Therefore a:(ln \ip' (x)\)' is a monotonically increasing function of x. 


> 0. 


(ii) Next, suppose ip(x) = \J y — 1. Then ip'(a 


1 


2X^ y/1 — 


:. Now we will evaluate (In \ip'(x)\)'. 


(In \ip'(x)\) r 


1 /3y / x( 1 — x) — x? 

2xi y/1 — x V y/1 — x 
Ax- 3 
2a;(l — x) 


This implies (x(In \ip'(x)\)') = 

dx dx 


( x 4x ~ 3 ) 

V 2(1 -x)xJ 


2(1 — a :)xJ 2(1 — x) 2 


> 0. 
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ViPQj) 


5. Denote h(P,qj,q n ) := ———Then — ' ’ ^ 0, where qj < q n < qi. 

tp'{pq n ) dp L hp(P, qi,q n ) J 


hp{P, qji q n ) 


(i) Suppose ^(-) = tanh(-). Then ip'{pqj) = - 

2/% \/l - /% 

hp{P,qj,q n ) 


. This implies h(P, qj ,q n ) = 


g„ /1 - /%, 


Qj V 1 - PQj 


Next, we will compute the quotient 


It is easy to see that hp(p,qj,q n ) = 


hp(P, qi,q n ) ’ 

q n Qj - Qn 


Therefore, 


Qj 7 Qn) 


2 qj V 1 - fiQn(l - PQj) 3/2 ' hp(P,qi,q n ) 

-y -^T’TT— f g; L /2 • Xt can be easil y checked that ± 0 for all (/3, q 3 , g„, qi) 

Qjiqi-Qn) (1 - PqjY /2 d/3\hp{P,qi,q n )J 

where gj < q n < qi- 

(ii) Now consider </>(•) = arctan(-). Then ip'(pq) = — - 


2 (pq)i \J 1 - /V 

Using the expression for ip'(Pq), we obtain h(fi,q.j,q n ) = f—) 2 ^ 

V qj J 


1 ^ PQj 


and h{fi,qi,q n ) = 


(-)' 

\qi' 


1 1 - ffgrc 

1 - Pdi 


. These are identical to case of <(>(•) = tanh(-). Remaining details can be 


checked in similar manner. 

6. For all p > q, we have ^ (Jjfg) < 0. 

(i) Consider </>(•) = tanh(-). 

Then - arctanh (\ /1 “ Pp) 


. Using definition of arctanh(-), we get 


ipifiq) arctanh( v / l — P q) 

*m = lEZEgl . Now we will compute 4 . 

w ,) ,„(l±^g) 


It is easy to see that ^ ( 


1 + y/1 - ftp 

'1 - \/l - Pp 


dP \if>(Pq)) 

1 

/3\A - Pv 


Therefore 


In 

'i+VT=^\ 

< i-\/i-/sp / 

In 

'l+Vi-09 \ 

k, l-y/l-Pq) 


Recall that arctanh(a;) = In ^ 


i + \/r~ 


'1 — \/l — X‘ 

arctanh( v / l — Pp) < arctanh( v / l — Pq)- This implies that 


)) 

J_In (+ 1 in ( 1+dUp 

\l-VT^qJ__JU/T^Q yi-V^P 

( 1WE5 V 

: j. Since p > q 1 we get 


1 > ,, 1 ■ , and 




/V1 - pq 

Hence ap ($Sy < °> where p > Q- 
(ii) Now, consider pp) = arctan(-). 


arctanh( yl — Pp) — 


pyj 1 - Pp 


arctanh( \/l — /3g) < 0. 
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Then 4 ’^ p) 
inen P(/3 q) 


■\J \_p P q • This implies that 


d f ipjpp) \ 
d/3 \ip(/3q)J 


q 1 1 - Pq /-p{l - @q) - {-q){l - Pp) 


P 2 V 1 - Pp 


(1 - /? g ) 2 


K 0,8i " ce ” =• 


7. For all a: > 0, function ip(-) satisfies the following property; 

(i) Consider </>(•) = tanh(-). For this case ip( x ) = arctanlr(v / l — x), and ip'(x) = 
We obtain 


2 xWl — X * 


d / ip(x) \ d / arctanh( ^/l — x) \ 
dx\xib'(x)J dx\ x --—/ 


^arctanh(vT — x) ^ 1 ^ arctanh(\/l — x) ^ 1 
V 2\/T — x 2xJ y/1 — x x 


Therefore 


d / d tp(x) \ d (x arctanh( -^/ 1 — x) 


+ 1 


) 


(x ( nX) j) = -(- 

dx\ dx xip'(x) ) dx\ y/1 — x 

1 -U _ p 

V \/l — x / 


2(1 — x) 


We know that ^-(arctanh(a:)) > 1 for all x such that 0 < x < 1, which, in turn implies that 
arctanh( \/l — x) 




> 1. Hence 


i ( r d ( V’(x) A n 
dx dx v x'ip'(x) ' J 


(ii) Now suppose </>(•) = arctan(-). Here ip(x) = ^ ^ — 1. Moreover ip’(x) 
We obtain 

d / ^(x) \ _ d r -- 3\ 

dx\xip'{x)) dx\ X\fx ) 

= 2 . 


2a:3 i/l^a 


Now, it is easy to see that £(*£(4%)) = 2 > 0. 


12 Conclusion 


We have studied existence of points of local maxima for the function f(x) = Ci</>(£i), over a hyper¬ 

plane. We have found conditions, imposed on the function <j>(-), which guarantee existence of at most 
one point of local maximum for the function on the hyperplane. Those conditions are satisfied by wide 
range of neuron transfer functions. The next step involves computing the points of local maxima for the 
nonlinear function over planes of lower dimension. 
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